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INDEPENDENT TECHNICAL REVIEW INTO THE LABOUR FORCE SURVEY AND 
ABS RESPONSE 


PURPOSE 

This paper presents the Executive Summary section of an independent technical review into 
Australian Bureau of Statistics (ABS) labour force estimates and the ABS response to the 
recommendations in the review. 


BACKROUND 


With the release of the September 2014 labour force estimates on 9 October 2014, the ABS 
announced an independent technical review would be conducted after unexpected volatility was 
observed in seasonally adjusted labour force estimates for July to September 2014. The review 
examined how the ABS produces the monthly labour force estimates to ensure that high quality, 
credible estimates will be produced. 


The review was undertaken in October and November 2014 by Paul McCarthy. Mr McCarthy 
worked in the ABS in a variety of senior management positions responsible for critical economic 
statistics and also worked for the OECD and World Bank on economic statistics. The review was 
guided by a reference committee consisting of senior officials from the Treasury, the Reserve 
Bank of Australia and the ABS. The final review report was provided to the Australian Statistician 
on 27 November 2014. 


The key recommendation of the review (recommendation 2) was implemented with the October 
2014 estimates released in Labour Force, Australia (cat. no. 6202.0) on 6 November 2014. 


REPORT FROM THE INDEPENDENT TECHNICAL REVIEW 


The Executive Summary section of Paul McCarthy’s independent technical review is in the 
Attachment. The full report from the technical review is available on request by emailing 
labourforce@abs.gov.au. 


RECOMMENDATIONS AND ABS RESPONSE 


Each recommendation from the independent technical review is presented below along with the 
ABS response to the recommendation. 


The ABS has accepted the findings of the review and has agreed to all the recommendations 
made. 


Of the 16 recommendations arising from the review, the ABS has already implemented two 
(recommendations 2 and 16), five are currently being implemented (1, 4, 7, 8 and 15) and 
another four (3, 5, 6 and 13) will be implemented in early to mid-2015. The remaining 
recommendations (9, 10, 11, 12 and 14) require more substantive investment or are conditional 
on future developments and will be included in relevant work plans. 


Increased number of respondents using e-forms 

Recommendation 1: The labour force characteristics of those responding by e-forms should 
continue to be analysed to determine how they compare with those being interviewed by 
telephone or in person and whether or not a changing take-up of e-forms could have an effect on 
the LFS estimates. 

ABS Response: Agree and is in progress. Findings will be reported in the February 2015 issue 
and then six monthly while e-form uptake is increasing. 


Changes made to the LFS supplementary survey program 

Recommendation 2: The LFS estimates should be seasonally adjusted using prior corrections 
for the supplementary surveys that have been assessed as having a significant effect on the 
seasonality of the LFS estimates from now on. The methodology should be closely examined as 
part of the annual seasonal reanalysis of LFS data in early 2015 to determine whether or not the 
method should be refined further prior to it being applied to the full length of LFS monthly series 
from February 1978 onwards. 

ABS Response: Agree and already implemented in the October 2014 labour force release in 
relation to seasonally adjusting labour force estimates using prior corrections for the 
supplementary surveys. Agree and will proceed to implement immediately in relation to 
examining the methodology as part of the annual seasonal reanalysis. 


Potential problems 

Recommendation 3: The ABS should warn users of LFS data of the possibility of instability in 
the seasonally adjusted LFS estimates in those months in which new supplementary surveys are 
being conducted from 2014 on and that it will be at least three years before the effects on the 
LFS estimates of the new supplementary survey program can be measured with any degree of 
certainty. Steps should be taken to identify any such effects (e.g. by analysing individual records 
that change LF status more than once) and adjust the core LFS estimates, if necessary. 

ABS Response: Agree and will proceed to implement immediately as part of the annual 
seasonal reanalysis. A statement will be included in the February 2015 publication. 


Lower response rates 

Recommendation 4: It is possible that the lower response rates may be having an effect on the 
estimates of employment and/or unemployment. The impact of lower response rates should be 
analysed further. 

ABS Response: Agree and is in progress. Findings will be reported in early 2015. The ABS 
remains committed to maintaining response rates in the 93-95% range. 


Trend estimates 

Recommendation 5: Expand the description of seasonal adjustment and trend in the 6202.0 
Explanatory Notes (and in any related publications) by including the links between an original 
series and its seasonal, irregular and trend components (i.e. O = T * S * I) as an introduction to 
the more technical aspects already included in the notes. 

ABS Response: Agree and will proceed to implement immediately. 


Gross flows 

Recommendation 6: The ABS should highlight the fact that the changes in the numbers of 
persons employed, unemployed and not in the labour force from one month to the next are the 
net outcomes of considerably larger gross flows between these categories. One means of doing 
so would be to include some comments on the gross flows each month in the “Labour Force 
Commentary” section of Labour Force, Australia (cat. no. 6202.0). 

ABS Response: Agree and will proceed to implement immediately. However, gross flows data 
currently only apply to a subset of the labour force estimates (i.e. the matched part of the 
common sample) and therefore the interpretability of the comments will need to be carefully 
considered. 


Bayesian techniques 

Recommendation 7: Bayesian techniques should be considered as a means of identifying and 
resolving potential problems in core LFS series. 

ABS Response: Agree and is in progress (in relation to assessing the relevance of the 
techniques) with findings to be reported by June 2015. 


Composite estimation 

Recommendation 8: An ongoing program should be established to systematically re-calculate 
the weights underlying composite estimation. The frequency of re-calculation should be based on 
an analysis of the effects of the changes in the weights between 2007 and the current time. 

ABS Response: Agree and is in progress with findings to be reported by April 2015. 


The LFS computing system 

Recommendation 9: The LFS system and associated collection systems need to be replaced so 
that proposed changes to the LFS can be formally assessed (e.g. through splitting the sample 
and comparing responses to new and old questionnaires). As an interim measure, a separate 
means of storing seasonal factors in the LFS system should be implemented so changed 
seasonal adjustment methods can be tested without impinging on the LFS production 
environment and that the seasonal factors from the SEASABS production system can be used 
selectively, if required. 

ABS Response: Agree but further work required before implementation to determine the 
feasibility and cost of the interim measure and system replacement. An update will be included 
with the March 2015 publication. 


ABS LFS Supplementary Surveys Program 

Recommendation 10: Given the importance of the LFS data and the adverse effect of the 
associated supplementary survey program on these key economic indicators, the ABS should 
consider discontinuing the supplementary surveys, or conduct them in a manner that has a 
negligible impact on the LFS data. 

ABS Response: Agree (to consider) and will be included in relevant work plans. Consultation 
with key stakeholders will be an integral component of the work plan. An update will be provided 
in mid-2015. 


Timing of release of LFS publications 

Recommendation 11: The LFS processing and publication cycle should be closely monitored to 
determine whether or not it is necessary to delay publishing LFS data until early in the week 
following the current timetable on an ongoing basis or at least when changes are being made to 
the LFS. In particular, delaying the publication should be considered if any significant change is 
to be implemented. 

ABS Response: Agree and will be included in relevant work plans in relation to future changes 
to the LFS. Needs further consideration, including consultation where appropriate, in relation to 
any ongoing change to the LFS publication cycle. An update will be provided in mid-2015. 


Stakeholder engagement 

Recommendation 12: The ABS should involve key stakeholders in any proposals to change the 
LFS or the ways in which LFS estimates are compiled. Any proposed changes should be 
communicated in advance via a release on the ABS website providing details of what is 
happening and why. 

ABS Response: Agree and will be included in relevant work plans. 


Volatility in the LFS estimates 

Recommendation 13: Statistics Canada has faced similar criticisms to the ABS regarding 
perceived volatility in its LFS estimates. One of the responses was to publish an article 
Interpreting monthly changes in employment from the labour force survey to show that recent 
history was, in fact, less volatile than the past. Although the recent volatility in the Australian LFS 


had a specific cause, the internal ABS analysis of the general volatility in the Australian LFS 
estimates should be expanded and formalised and published to help educate users about this 
issue. 

ABS Response: Agree and will proceed to implement immediately. An article will be released by 
April 2015. 


Making changes to the LFS 

Recommendation 14: Wherever feasible, future changes to the LFS program should be 
introduced individually, spread out over time rather than bunched as has been the case during 
the past 18 months or so, to the extent they can be controlled (e.g. the timing of introducing 
Population Census benchmarks and benchmarking to ERP estimates is non-discretionary). 
ABS Response: Agree and will be included in relevant work plans. Changes will be spread over 
time unless the benefits of more rapid change, assessed through consultation with key 
stakeholders, are considered compelling. 


Change management 

Recommendation 15: If it does become necessary to make a number of changes to the LFS in 
a short time, specific risk management strategies need to be put in place to assess the potential 
impacts of each of the changes. A small working group consisting of senior Labour Branch/LFS 
staff and representatives of Methodology Division, Time Series Analysis Branch and Technology 
Services Division should evaluate the individual and cumulative impacts of the proposed changes 
and provide a formal risk management report to the relevant FAS and Deputy Statistician. 

ABS Response: Agree and will be included in relevant work plans. In preparation, the ABS’s 
methodology for managing statistical risk during change will be reviewed and updated by the end 
of 2014. 


Quality Incident Response Plan 

Recommendation 16: As part of any future Quality Incident Response Plan (QIRP), the effects 
on LFS seasonality of changes in institutional arrangements (e.g. changes in the LFS 
questionnaire or in the supplementary survey program or in the incidence of response by 
e-forms) should be considered explicitly. 

ABS Response: Agree and already implemented in the relevant template. 


ATTACHMENT - EXECUTIVE SUMMARY OF THE INDEPENDENT TECHNICAL 
REVIEW INTO THE ABS LABOUR FORCE SURVEY 


EXECUTIVE SUMMARY 
Introduction 


In July, August and September 2014, the ABS Labour Force Survey (LFS) seasonally adjusted 
employment and unemployment data have been unusually volatile. As a result, in the September 
2014 LFS results, the ABS set the seasonal factors for each of these three months to one (i.e. 
assuming no seasonality in these months). 


Following the release of the September LFS estimates, the ABS announced that “it would 
commission a review with independent external input to develop an appropriate method for 
seasonally adjusting October 2014 and following months’ estimates”. 


The scope of this independent review for the acting Australian Statistician’s consideration was to 
identify issues that impacted on the quality of recent LFS estimates, particularly any related to the 
changes that have been made to the LFS over the past 18 months or so, with emphasis on 
advising an appropriate course of action for changes required when compiling the October 2014 
LFS estimates. Recommendations regarding any changes required to ensure the quality of LFS 
estimates in subsequent months were also to be included. 


Recent changes to the LFS 


Several significant changes have been implemented in the LFS program during the past 18 
months: 


1. Introducing 2011 Population Census benchmarks into the LFS frame 


2. Benchmarking the LFS to revised Estimated Resident Population estimates (20 
years data were affected) 


3. Changes to the questions on “job search” 
4. Other changes to the LFS questionnaire 
5. Changes to the LFS supplementary survey program 
6. Introducing and expanding the use of e-forms 
7. Response rates were lowered. 
Effects of the changes 


The effects of the first four are judged to be minimal, apart from the potential for some noise in 
the data at the time of their introduction. 


The changes to the supplementary survey program have been the major reason for the volatile 
LFS estimates, particularly in July, August and September 2014 but also in some other months 
earlier in the LFS time series. The problem was that some supplementary surveys have an effect 
on the responses to the main LFS survey (e.g. the supplementary survey run in August for many 
years is associated with a lower level of employment than should be the case). 


Recommendations regarding the October 2014 LFS estimates 


The review found that the major source of volatility in the LFS estimates in recent months was 
the change in the supplementary survey program that is run in conjunction with the LFS. Other 
potential sources of volatility were the move to e-forms for the LFS and lower response rates, 
although these were considered to be marginal compared with the effects of the changed 
supplementary survey program. 


The key recommendation regarding the October 2014 LFS estimates was provided ina 
preliminary report on 29 October 2014: 


The LFS estimates should be seasonally adjusted using prior corrections for 
the supplementary surveys that have been assessed as having a significant 
effect on the seasonality of the LFS estimates from now on. The methodology 
should be closely examined as part of the annual seasonal reanalysis of LFS 
data in early 2015 to determine whether or not the method should be refined 
further prior to it being applied to the full length of LFS monthly series from 
February 1978 onwards. 


The ABS agreed to this recommendation and, for the October 2014 LFS, implemented a 
seasonal adjustment method that treated the effects of supplementary surveys as “prior 
corrections” in the seasonal analysis of LFS (i.e. they were identified separately from the 
seasonal influences rather than as part of the seasonal effects). The ABS has also agreed to 
refine the methodology as part of the annual seasonal reanalysis of LFS estimates in early 2015. 


Outcomes of the revised seasonal adjustment methodology 


Seasonal adjustment is the process of estimating and then removing from a time series those 
influences that are calendar related, i.e. those that tend to occur regularly once or more each 
year. They include the effect of periodic factors such as weather, holidays, social conventions, 
administrative practices and, in some indicators, the number of trading days in each month. 
Seasonally adjusted estimates are produced by estimating the seasonal component and 
removing it from the original (unadjusted) series. 


There is evidence that LFS supplementary surveys have a systematic influence on the LFS time 
series. Historically, the supplementary surveys program has been consistent in the sense that, in 
general, the same supplementary surveys have been conducted annually in the same month. 
The reasonably consistent timetable for major supplementary surveys means that any effects 
they might have on the LFS data have generally been identified as part of the seasonal 
adjustment process and so have been removed from the seasonally adjusted data. 


Starting in February 2014, the timetable for supplementary surveys changed quite dramatically. 
As aresult, the supplementary survey effect is no longer being removed as part of the ongoing 
seasonal adjustment process. There was no obvious effect on the seasonality of the LFS 
employment or unemployment in the first few months after the change in this timetable, although 
the February LFS estimates were considered to be a bit noisier than usual. However, the LFS 
estimates became more volatile from July through to September. The seasonality observed in 
total employment and unemployment in these months was attributable to actual seasonal 
(calendar) influences plus the (unknown) extent to which the seasonality was caused by the 
supplementary surveys. Not adjusting for this latter factor when the supplementary survey 
program was changed would increase the effects of the irregular component of the time series; in 
other words, it would increase the volatility of the series. 


The effect of individual supplementary surveys on the original (unadjusted) LFS estimates was 
tested using regression analysis. Seven supplementary surveys were found to most significantly 
influence the employment estimates and those seven plus an additional six were identified as 
having a significant influence on the unemployment estimates. 


Prior corrections to take account of the effects of these supplementary surveys were estimated at 
the national level and applied to each of the directly seasonally adjusted components of 
employed persons and unemployed persons. Using the prior corrections to explicitly estimate the 
effects of the LFS supplementary surveys is judged to have provided a more reliable set of 
seasonally adjusted estimates and will be used in the ongoing seasonal adjustment of LFS 
estimates. 


As an interim measure, the new seasonal adjustment methods were applied to data from 
December 2013 onwards. The full length of the monthly LFS time series (i.e. from February 1978 
onwards) will be adjusted using this method when the annual seasonal reanalysis of the LFS is 
completed in early 2015. 


Potential risks 
There are three potential influences on the LFS estimates that need to be closely monitored: 


1. The influence of the supplementary surveys on the LFS could change again as new 
supplementary surveys are introduced or existing ones are amalgamated. 


2. Introducing and expanding the use of e-forms could have an effect on the LFS 
estimates. 


3. Lower response rates could potentially bias the results or, at least, add volatility 
due to increases in the relative standard errors. 


In addition, any proposals for making significant changes to the LFS (as has occurred in the past 
18 months) should be subjected to a formal change management risk assessment. 


Changes caused by new supplementary surveys 


The effects of discontinuing some supplementary surveys can be assessed using regression 
analysis and treated as prior corrections in the seasonal analysis of LFS data. However, there is 
no information available on the effects of introducing new supplementary surveys or of combining 
some Surveys previously conducted separately. Therefore, the new program currently being 
introduced will potentially affect the LFS estimates to an unknown extent for at least the next 
three years. Any effects will show up as increased volatility in the seasonally adjusted LFS 
estimates in the months in which the new supplementary surveys are conducted and in the 
movements between those months and the adjoining ones. 


One means of determining if there is a “supplementary survey effect” on the LFS would be to 
identify records that change their status in the month of a supplementary survey and then change 
back again in the following month. For example, a person classified as employed in the month 
before the supplementary survey, switching to unemployed in the supplementary survey month 
and then back to employed in the following month could be followed up to ascertain whether the 
changes were valid. 


Alternatively, a more broadly based approach would be to check the history of numbers who 
have hopped from one category to another then back to the previous category across a major 
supplementary survey month. For example, for August where the observed supplementary 
survey effect is lower employment, the analysis would involve checking the numbers who were 
employed in July, changed to either unemployed or not in the labour force in August and then 
back to employed in September. Running a regression through the historical data would give the 
expected numbers of category hoppers, which could be compared with those in August this year 
to provide an indication of the changed impact from the new August survey. The seasonal factors 
for August could then be adjusted to take account of this assessment of the changed 
supplementary survey effect due to the new August survey. One problem is that the full extent of 
the category hoppers cannot be worked out until after the September LFS data are available. In 
August 2015, for example, an initial estimate of the effect could be made using 2014 data and 
then the seasonal factors for August 2015 could be revised once the full analysis can be carried 
out using the September LFS data. A similar analysis should be tested for each of the months in 
which a supplementary survey was run that has been shown to have an effect on the LFS 
results. 


Statistics Canada may have a process for handling this type of effect and it would be worthwhile 
to check if their procedures could be adopted. 


Expanding the use of e-forms 


The ABS introduced e-forms into the LFS in May 2013. The current response by e-form has been 
fairly stable for several months at about 20% but the aim is to increase this level significantly. 
During the phase-in of online forms it was noted that employed persons were more likely to use 
the e-form option than unemployed persons or persons not in the labour force. However, there 
was no indication that persons using e-forms would respond differently to the questions 
compared with those being interviewed by phone or in person. 


Introducing the e-forms is likely to cause two types of effect — a coverage effect and a reporting 
effect. The coverage effect is when the e-form captures a different respondent population than 
the existing paper form. The reporting effect is any tendency for respondents to provide different 
answers according to whether they are responding to an interviewer or using an e-form. The 
e-forms were introduced using a “split sample” approach, which enabled an analysis to separate 
these two effects by comparing the distribution of labour force status between the e-form 


respondents and similar respondents from the control group who were not offered the e-form 
option. The analysis showed that there was evidence of a coverage effect but there was no 
measurable reporting effect given both the known limitations of the strategy to detect a small 
impact and the relatively low e-form take-up rate. 


Lower response rates 


The LFS response rates have fallen from about 96% to 93% between the beginning of 2013 and 
September 2014. Analysis of the final 2-4% of responses in earlier years indicated that these last 
responses were not significantly different from the initial response. As a result, it was concluded 
that a drop in the response rates from 95-96% to about 93-95% should not have a significant 
effect on the LFS estimates. However, the response rates have fallen to the low end of that 
bound. 


An analysis of the unmatched part of the common sample in September 2014 showed that the 
missing sample in August due to the fall in response rates was a group that was less likely to be 
employed. Labour Branch analysis showed that those who did not respond in August were 
different from the usual “hard to gets” which are more likely to be employed. They were likely to 
be either unemployed or not in the labour force, which would cause a higher employment 
estimate and lower unemployment estimate. (The response rates had dipped from about 93% in 
July to about 92% in August and then rose to about 934%2% in September.) 


Further evidence that the decline in response rates is potentially having an adverse impact on the 
survey results is that the preliminary estimates for August 2014 changed significantly in the few 
days between the preliminary estimates on Monday 1 September and the final estimates, which 
became available on Friday 5 September. However, part of this effect could be due to delays in 
coding records that had been received earlier. 


There is no indication that the lower response rates are biasing the LFS results; rather, if they are 
having an effect then it is likely to show up as increased noise in the estimates. It should be 
noted that the Australian LFS response rates are still very high by international standards. For 
example, LFS response rates in Canada were around 93% prior to 2012 but have dropped to 
about 88-90% in the past couple of years. The response rates for the United Kingdom’s LFS are 
less than 80%. 


Change management 


The changes made to the LFS during the past 18 months or so placed a huge load on the LFS 
staff. Key personnel left part way through the process. Their replacements have performed well 
in difficult circumstances. However, LFS users are interested in the quality (believability!) of the 
LFS data rather than any problems being encountered by the ABS in handling the range of 
changes involved. A more systematic and comprehensive approach (including regular reviews) to 
managing statistical risks around the introduction of changes should be adopted. Future changes 
to the LFS should proceed only after a formal risk assessment is carried out on all aspects of the 
proposed changes. All relevant areas of the ABS should be involved (Labour Branch, 
Methodology, Time Series Analysis, Technology Services (if required)) and they should formally 
sign off an assessment of the effects of the changes and how they will be managed. This is 
particularly important where the change is complex, where change occurs over a long period of 
time, where ‘outside factors’ cause plans to change or when there are changes in key personnel. 


Stakeholder engagement 


The LFS data are amongst the more important economic indicators produced by the ABS. Any 
changes to these data and the ways in which they are compiled need to be communicated to key 
stakeholders well in advance of the changes being introduced. It is important to provide details of 
the reasons for any changes together with an assessment of their effects on the estimates. 


SUMMARY OF RECOMMENDATIONS 


This section lists the recommendations in the report and the paragraph numbers of the text to 
which each relates (note that there are gaps in the paragraph numbers because not every 
paragraph leads to a recommendation). 


Increased number of respondents using e-forms (paragraphs 26-31) 


Recommendation 1: The labour force characteristics of those responding by e-forms should 
continue to be analysed to determine how they compare with those being interviewed by 
telephone or in person and whether or not a changing take-up of e-forms could have an effect on 
the LFS estimates. 


Changes made to the LFS supplementary survey program (paragraphs 32-51) 


Recommendation 2: The LFS estimates should be seasonally adjusted using prior corrections 
for the supplementary surveys that have been assessed as having a significant effect on the 
seasonality of the LFS estimates from now on. The methodology should be closely examined as 
part of the annual seasonal reanalysis of LFS data in early 2015 to determine whether or not the 
method should be refined further prior to it being applied to the full length of LFS monthly series 
from February 1978 onwards. 


Potential problems (paragraphs 52, 53) 


Recommendation 3: The ABS should warn users of LFS data of the possibility of instability in 
the seasonally adjusted LFS estimates in those months in which new supplementary surveys are 
being conducted from 2014 on and that it will be at least three years before the effects on the 
LFS estimates of the new supplementary survey program can be measured with any degree of 
certainty. Steps should be taken to identify any such effects (e.g. by analysing individual records 
that change LF status more than once) and adjust the core LFS estimates, if necessary. 


Lower response rates (paragraphs 54-57) 


Recommendation 4: It is possible that the lower response rates may be having an effect on the 
estimates of employment and/or unemployment. The impact of lower response rates should be 
analysed further. 


Trend estimates (paragraphs 64-68) 


Recommendation 5: Expand the description of seasonal adjustment and trend in the 6202.0 
Explanatory Notes (and in any related publications) by including the links between an original 
series and its seasonal, irregular and trend components (i.e. O = T * S * I) as an introduction to 
the more technical aspects already included in the notes. 


Gross flows (paragraphs 69-71) 


Recommendation 6: The ABS should highlight the fact that the changes in the numbers of 
persons employed, unemployed and not in the labour force from one month to the next are the 
net outcomes of considerably larger gross flows between these categories. One means of doing 
so would be to include some comments on the gross flows each month in the “Labour Force 
Commentary” section of Labour Force, Australia (cat. no. 6202.0). 


Bayesian techniques (paragraphs 72-74) 


Recommendation 7: Bayesian techniques should be considered as a means of identifying and 
resolving potential problems in core LFS series. 


Composite estimation (paragraphs 75-78) 


Recommendation 8: An ongoing program should be established to systematically re-calculate 
the weights underlying composite estimation. The frequency of re-calculation should be based on 
an analysis of the effects of the changes in the weights between 2007 and the current time. 


The LFS computing system (paragraphs 79-82) 


Recommendation 9: The LFS system and associated collection systems need to be replaced so 
that proposed changes to the LFS can be formally assessed (e.g. through splitting the sample 
and comparing responses to new and old questionnaires). As an interim measure, a separate 
means of storing seasonal factors in the LFS system should be implemented so changed 
seasonal adjustment methods can be tested without impinging on the LFS production 
environment and that the seasonal factors from the SEASABS production system can be used 
selectively, if required. 


ABS LFS Supplementary Surveys Program (paragraphs 83) 


Recommendation 10: Given the importance of the LFS data and the adverse effect of the 
associated supplementary survey program on these key economic indicators, the ABS should 
consider discontinuing the supplementary surveys, or conduct them in a manner that has a 
negligible impact on the LFS data. 


Timing of release of LFS publications (paragraphs 84-87) 


Recommendation 11: The LFS processing and publication cycle should be closely monitored to 
determine whether or not it is necessary to delay publishing LFS data until early in the week 
following the current timetable on an ongoing basis or at least when changes are being made to 
the LFS. In particular, delaying the publication should be considered if any significant change is 
to be implemented. 


Stakeholder engagement (paragraphs 88, 89) 


Recommendation 12: The ABS should involve key stakeholders in any proposals to change the 
LFS or the ways in which LFS estimates are compiled. Any proposed changes should be 
communicated in advance via a release on the ABS website providing details of what is 
happening and why. 


Volatility in the LFS estimates (paragraph 103) 


Recommendation 13: Statistics Canada has faced similar criticisms to the ABS regarding 
perceived volatility in its LFS estimates. One of the responses was to publish an article 
Interpreting monthly changes in employment from the labour force survey to show that recent 
history was, in fact, less volatile than the past. Although the recent volatility in the Australian LFS 
had a specific cause, the internal ABS analysis of the general volatility in the Australian LFS 
estimates should be expanded and formalised and published to help educate users about this 
issue. 


Making changes to the LFS (paragraphs 104-106) 


Recommendation 14: Wherever feasible, future changes to the LFS program should be 
introduced individually, spread out over time rather than bunched as has been the case during 
the past 18 months or so, to the extent they can be controlled (e.g. the timing of introducing 
Population Census benchmarks and benchmarking to ERP estimates is non-discretionary). 


Change management (paragraphs 107-110) 


Recommendation 15: If it does become necessary to make a number of changes to the LFS in 
a short time, specific risk management strategies need to be put in place to assess the potential 
impacts of each of the changes. A small working group consisting of senior Labour Branch/LFS 
staff and representatives of Methodology Division, Time Series Analysis Branch and Technology 
Services Division should evaluate the individual and cumulative impacts of the proposed changes 
and provide a formal risk management report to the relevant FAS and Deputy Statistician. 


Quality Incident Response Plan (paragraphs 111-114) 


Recommendation 16: As part of any future Quality Incident Response Plan (QIRP), the effects 
on LFS seasonality of changes in institutional arrangements (e.g. changes in the LFS 
questionnaire or in the supplementary survey program or in the incidence of response by 
e-forms) should be considered explicitly. 
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